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Description 

Method and device for three-dimensionally detecting objects and the 
use of this device and method 

The invention relates to a method for three-dimensionally detecting 
objects, in which 

- a color pattern with known projection data is projected onto the 
object to be detected, 

- the pattern projected onto the object to be detected is recorded 
with the aid of a camera and 

- the image created by the camera is processed in an analysis unit 
into three-dimensional object coordinates. 

The invention further relates to a device for executing the method 
and to use of the device and the method. 

Methods for three-dimensional detection of objects are needed for 
different purposes, including the identification and authentication 
of persons. Thus for example projecting a two-dimensional color 
pattern made of colored pattern elements and using a camera to 
record the color pattern projected onto the object to be detected is 
known from DE 199 63 333 Al . As a result of the spatial properties 
of the surface of the object to be detected the color patterns in 
the image recorded by the camera are shifted in relation to the 
original arrangement, so that for a known position of the projector 
and the camera, the three-dimensional data of an object point on the 
surface of the object to be detected can be calculated. 

The known method is however not suitable for colored scenes. 
' Instead, with the known method, it is necessary to record a 
reference image with evenly white object illumination. This 
reference image allows the projected colors in the sample image to 
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be determined. Since with the known method at least two images need 
to be recorded, it is limited as to its suitability for moving 
objects or objects which change their shape. Thus this known method 
has only restricted suitability for detecting a hand or a face. 

5 Projection of a black and white two-dimensional pattern with 

additional encoding into the object to be detected is known from the 
publication by P. Vuylsteke and A. Oosterlinck, "Range Image 
Acquisition with a Single Binary-Encoded Light Pattern" , IEEE 
Transactions on Pattern Analysis and Machine Intelligence, Volume 

10 12, No. 2, February 1990. This attempts to determine the angle of 
projection for each point of the camera image from the image 
environment of the point. In particular it attempts to calculate the 
angle of projection of the relevant point from the distortions of 
the projected pattern arising from the spatial orientation of the 

15 object surface as viewed by the camera. However -this only succeeds 
well for objects with a constantly moving surface. If however 
indentations are present on the object to be detected, the coding 
pattern elements of the pixel environment are disturbed, which 
results in non-determinable or incorrect projector angles and leads 

20 as a result to incorrect three-dimensional object coordinates. 

Using this prior art as its point of departure, the object of the 
invention is thus to specify a method for three-dimensional 
detection of objects which is not adversely affected either by the 
coloration or by indentations in the object to be detected. 

25 A further object of the invention is to specify a device for 
executing the method. 

These objects are achieved by a method and a device with the 
features of the independent claims . Advantageous embodiments and 
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developments of the invention are specified in dependent claims of 
said claims. 

With the method in accordance with the invention the projection 
coordinates are encoded in the color pattern with the aid of a 
5 redundant code. The use of a color pattern for encoding offers the 
advantage of enabling encoding to be executed in a compact way such 
that damage to the code by indentations or covered areas, especially 
in facial recognition, is unlikely. Above and beyond this the 
encoding is robust as regards errors since color discrepancies in 
10 the image generated by the coloring of the object can be detected 
and eliminated in the image. 

A device set up to execute the method thus features a projector 
which projects a redundantly coded color pattern onto the object to 
be detected. 

15 In a preferred embodiment of the method the color values of the 

color pattern are subdivided into codewords. The codewords in this 
case preferably exhibit a non-trivial Hamming distance. A non- 
trivial Hamming distance should in this case be taken to be a 
Hamming distance greater than 1. With this, form of encoding good 

20 results in the detection of the encoded projection data have been 
achieved . 

The robustness of the encoding can be further increased by having 
the codewords overlap. 

Further measures relate to the color changes in the individual 
25 channels . The detection accuracy can for example be increased by the 
color values in each color channel being able to assume only two 
color values defined in advance. Expediently the color change ranges 
in each color channel between a minimum possible value and a further 
maximum possible value. 
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A further requirement can be imposed for color changes to have to 
occur simultaneously in at least two color channels. 

Errors attributable to changes in the coloring of the object to be 
detected can be eliminated by these two methods. 

5 A further useful requirement for the color pattern relates to the 
number of color changes in each codeword. If at least one color 
change takes place in each codeword in each of the color channels, 
detection errors cannot be propagated over a number of codewords. 

The detection accuracy can be further improved by encoding the color 
10 changes into codewords of a redundantly encoded code. These 

requirements for the encoding of the color pattern allow errors in 
decoding to be reliably detected. 

Decoding is undertaken here by the analysis unit detecting the 
turning points of the detected measurement signal by determining the 

15 maxima of the first derivation. The turning points detected in this 
way, which can be assigned to potential color changes, are 
subsequently checked for the criteria which fulfill the encoding of 
the color code. This includes checking the size of the color change 
and the correlation of the color changes between the channels. A 

20 further criterion is for example that the turning points with 

positive and negative rise must each occur in pairs and that the 
color changes must correspond to the codewords of the color changes. 

Finally the color changes are checked for matches with the encoded 
codewords of the color change and the projection data assigned to a 
25 point of the recorded image. From the projection data and the 
coordinates of the relevant point in the image recorded by the 
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camera the three-dimensional object coordinates of the surface of 
the object to be detected can then be calculated. 

The invention is explained in detail below with reference to the 
enclosed drawing. The diagrams show: 

5 Figure 1 A view of a device for the three-dimensional 

recording of object data; 

Figure 2a to c Diagrams which illustrate the data reduction; and 

Figure 3a and b Illustrations of the reconstructed profile lines 

of a recorded object. 

10 Figure 1 shows a device 1 which is used to determine the three- 
dimensional object coordinates of a surface 2 of an object 3 to be 
recorded . 

Device 1 features a projector 4 which projects a color pattern 5 
onto the surface 2 of the object to be recorded 3 . In the case shown 
15 in Figure 1 the color pattern 5 is composed of a series of adjacent 
colored stripes . It is however also conceivable to use a two- 
dimensional color pattern instead of the one-dimensional color 
pattern 5 shown in Figure 1 . 

With the exemplary embodiment shown in Figure 1 each point P of the 
20 surface 2 of the object 3 can be assigned a projection level g. 

Projection data is encoded by the color pattern 5 in this way. The 
color pattern 5 projected onto the surface 2 of the object 3 is 
converted by a camera 6 into an image 7 in which the point P on the 
surface 2 is transformed into the point P' . For a known arrangement 
25 of the projector 4 and the camera 6, especially with a known length 
of the basic distance 8, triangulation can be used to calculate the 
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three-dimensional space coordinates P on the surface 2 . The data 
reduction and analysis required for this is performed by an analysis 
unit 9. 

To enable the three-dimensional space coordinates of the point P to 
5 be determined on the surface 2 from an individual image 7 even if 
the surface 2 of the object 3 has indentations and covered areas, 
the color pattern 5 is constructed in such as way that the encoding 
of the projection levels g is as robust as possible in relation to 
errors . The coding further allows errors which relate to the 
10 coloring of the object to be eliminated. 

For the exemplary embodiment shown in Figure 1 the colors of the 
color pattern 5 are described by the RGB model . The color values of 
the color pattern 5 are changed by changes to the color values in 
the individual color channels R, G and B. 

15 The color pattern should satisfy the following conditions : 

- Only two color values will be used in each color channel. In 
particular the minimum value and the maximum value respectively 
will be used in each color channel, so that in the RGB model 
overall a total of eight colors are available. 

20 - Within a codeword each color channel features at least one color 
change. This condition makes it possible to decode the individual 
codewords . 



25 



- Adjacent color elements differ in at least two color channels. 
This condition is used especially to guarantee error tolerance, 
especially in relation to indentations . 
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- The individual codewords of the color pattern 5 features a non- 
trivial Hamming distance. This condition too serves to increase 
the error tolerance on decoding of the projection levels g. 

- The color changes are also compiled into codewords with a non- 
5 trivial Hamming distance. 

An example is given below for color pattern 5 which satisfies the 
five conditions stated above. This color pattern 5 relates to the 
RGB model with a red color channel R, a green color channel G and a 
blue color channel B. Since color values in each color channel can 
10 only assume the minimum value and maximum value respectively, a 
total of eight mix colors are available to which the following 
numbers will be assigned: 





Black 


0 




Blue 


1 


15 


Green 


2 




Cyan 


3 




Red 


4 




Magenta 


5 




Yellow 


6 


20 


White 


7 



A length of four color stripes has been selected for the codewords 
of the color values, with adjacent codewords overlapping with three 
color strips. 

Numerical values have also been assigned to color changes. Since in 
each of the three color channels the color value can remain the 
same, can fall or can rise, a total of 27 different color changes of 
the mix colors is produced, each of which has been assigned a number 
between 0 and 26. The length of the codewords assigned to the color 
changes has been selected as equal to three color changes, with 
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adjacent codewords overlapping with two color changes . 

A search algorithm was used to find the following number series 
which describes an exemplary embodiment of the color pattern 5 which 
satisfies the above-mentioned five conditions: 

5 12 4307056121741427034212721653417161436160530 63 52717072416305 
25074714706503 5603 634743 506172 5242 53 607 

In the specified exemplary embodiment the first codeword consists of 
the digits 1243, the second codeword of the digits 2340 and the 
third codeword of the digits 4307. The exemplary embodiment shown 
10 represents a very robust encoding. 

Figures 2a to c illustrate the data reduction. Figure 2a shows an 
original color signal 10. The original color signal 10. is shaped by 
the surface 2 of the object 3 into a measurement signal 11. To 
clarify matters it should be pointed out that the color signal 10 
15 and the measurement signal 11 are shown running at an angle to the 
colored stripes of the color pattern 5. shown in Figure 1. 

For data reduction a first derivation 12 of the measurement signal 
11 is calculated in a first step. Subsequently the turning points of 
the measurement signal 11 are determined by calculating the extreme 

20 values 13 of the first derivation 12. The exact position is 

calculated by directly fitting a parabola or a cubic polynomial to 
the environment of an extreme value of derivation 12. The extreme 
values produced in this way are combined in accordance with their 
spatial distribution to form extreme values of the mixed color 

25 signal which extend over a number of color channels . The position of 
the extreme values extending over a number of color channels is 
calculated by forming a weighted average of the position of the 
extreme values in the individual color channels . The factors used 
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for weighting are formed on the basis of the quality of the fit 
curve and the signal-to-noise ratio of measurement signal 11. 
Individual extreme values which do not correspond to a further 
extreme value in another color channel are ignored below. Since the 
5 likelihood of the occurrence of an extreme value without 

corresponding color transition is very small, this is an effective 
means of eliminating noise-related distortions. In Figure 2c for 
example the mistakenly detected extreme value 14 can be eliminated. 

In a further procedural step identical extreme values will be 
10 combined into profile lines 15. Figure 3a shows the result for the 
case in which the extreme values are not subject to any further 
filtering. Since however the color changes are structured with the 
aid of overlapping codewords which have a non-trivial Hamming 
distance between them, those extreme values which are based on color 
15 edges in the coloration of surface 2 can also be filtered out. The 

result is shown in Figure 3b. The method described here is thus very 
much in a position to record the actual profile lines 15 of the 
color changes which are based on color changes in the color pattern 
5. 

20 In the subsequent execution sequence the projector levels g can be 

assigned via color coding to the points P of the image 7. The three- 
dimensional object data of the surface is then obtained by a 
triangulation, when the arrangement of the camera 6 and of the 
projector 4 are known. 

25 The method described here can also be used to determine the coloring 
of the object to be recorded 3. The color value of a point. P in the 
image 7 can be described by the following formula: 



K =r c js c (A) • /„ ( X)dX +r c -j s c (A) • /„ ( A)dA 
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where s c (A) gives the spectral sensitivity of a sensor element in the 
camera 6, r c (A) is the spectral reflectivity of the associated 
surfaces 2, I P (A) the spectral intensity of the radiation emitted by 
the projector 4 and I a (A) the intensity of the background 
5 illumination. Since I P (A) is known for each point P after the 

execution of the method described above, assuming that Rc(A) is 
essentially equal to a constant r c , it is possible to calculate from 
the measured value R c in each color channel the constant r c , when the 
effect of the background illumination can be ignored. The coloring 
10 of the surface 2 can then be obtained from the reflectivities r c of 
the three color channels. Taking account of adjacent values of r c it 
is finally possible to at least record the lower- frequency 
components of the coloring of the object 3. 

The method has already been extensively tested. A color pattern, in 
15 which 104 stripes formed a unique encoding was used. For the 

projected color pattern 5 however a total of 156 light levels was 
used since the geometry of the test setup allows partial repetition 
of the coded pattern. 

Both a simple slide projector and also an LCD projector were used 
20 for the projector 4. A simple slide projector for which the optics 
is optimized to a large depth of focus is the best alternative as 
regards cost. 

Various RGB cameras with a resolution of 574 x 768 pixels were used 
for the camera. To suppress any possible crosstalk between the 
25 individual color channels good color separation in camera 6 is 

essential. A computer with a Pentium IV 2 , 4 GHz processor was used 
for analysis unit 9. This allowed up to 15 images per second to be 
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recorded and analyzed without optimizing the speed of the program 
code. The method is thus entirely suitable for recognition of 
gestures . 

The method operates reliably if the variation of the coloring of 
5 surface 2 and the background illumination do not correspond in 

frequency and amplitude to the color change of the color pattern 5. 
This can also be seen from Figure 3a and 3b, in which the fingers 16 
are resting on a multicolored support 17. 

A surface must be at least 8x8 pixels in size in the image 7 to 
10 enable a three-dimensional coordinate to be assigned to it. Then at 
least two complete colored stripes are needed for analysis, with one 
colored stripe in camera 6 being at least 3 pixels wide; 

Since an individual recording of the surface 2 is sufficient for the 
execution of the method described here in order to determine three- 

15 dimensional coordinates of the surface 2, the method and the device 
described here are particularly suitable for recording moving 
objects or objects which change their shape. The method described 
here is thus especially suited within the context of biometrics to 
facial recognition and to recording gestures. To this extent the 

20 method described here is primarily suited to identification and 
authentication of people. 

Further areas of application relate to quality assurance of 
products, calibration of objects in the construction, for example 
for retrofitting, for repairs or for the expansion of existing 
25 machines or plant, or the three-dimensional modeling of objects in 
the multimedia and games area. 

It should be pointed out that other color models, e.g. the YUV 
model, can be used for the description of the color pattern 5. 
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It should also be pointed out that the term camera can be taken to 
mean any imaging system. 



